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Abstract 



The SAT Program tests are designed for use with juniors and seniors in high school as 
part of the college admissions process. At the time of registration, students voluntarily 
complete the Student Descriptive Questionnaire (SDQ) that asks a variety of questions, ranging 
from course-taking patterns, college preferences, self-reported grades, and various family and 
self-indicators. One of the questions on the SDQ asks students to choose the ethnic/racial 
category that describes them; however, only one ethnic/racial category may be chosen. 

The SAT Program undertook two studies aimed at evaluating the impact of allowing 
students to indicate more than one ethnic/racial category. Information was collected during 
two large fall administrations of the SAT I. Students were asked, immediately following 
testing, to indicate their ethnic/racial background on their answer sheet; multiple selections 
were allowed. During the first fall administration, students were asked to indicate aN 
categories that described them using essentially the same list found on the SDQ. During the 
second administration, students were asked to describe themselves using a two-part question, 
first indicating their ethnicity (Hispanic or non-Hispanic) and second their race (using the 
remaining SDQ categories). Approximately 93% of test takers responded to the questions on 
both the answer sheet and the SDQ and about 4.5% moved from a single to multiple 
ethnic/racial categories. The effect of the shift from single to multiple categories was further 
examined by comparing differential item functioning (DIF) results using the test takers from the 
first administration. Results indicated that the mean DIF changed very little when the groups 
were redefined using their responses on the answer sheet. These analyses provide preliminary 
information on the impact of allowing students to choose multiple ethnic/racial categories. 
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The SAT Program tests (SAT I: Reasoning Test and the SAT II: Subject Tests ) are 
designed for use with juniors and seniors in high school and scores are used to assist in the 
college admissions process. The Student Descriptive Questionnaire (SDQ) is the primary 
mechanism for gathering demographic and background information on students who take these 
tests. This questionnaire is voluntarily completed at the time students register to take the SAT 
I or SAT II and asks a variety of questions, ranging from course-taking patterns, college 
preferences, self-reported grades, and various family and self indicators. One of the questions 
on the SDQ asks students to choose the ethnic/racial category that describes them (see Table 
1). However, only one ethnic/racial category may be chosen. Even though responses to the 
SDQ are voluntarily, most questions have a high response rate; for example, the ethnic/racial 
question has over a 90% response rate. 

As student demographics have changed, increasing requests have been made to allow 
students to choose multiple ethnic/racial categories. In addition, in November 1997, the Office 
of Management and Budgeting issued changes to the racial and ethnic categories that allow for 
multiple selections. And, the Census 2000 allowed individuals, for the first time, to choose 
multiple racial and ethnic categories. As a result of these changes, the SAT Program decided 
to undertake a series of studies to examine the impact of allowing students to indicate more 
than one ethnic/racial category. 



3 




4 



Sample 



Data were collected information during two large fall administrations of the SAT I in 
1998. Students were asked, following testing, to indicate their ethnic/racial background on 
their answer sheet. Multiple selections were allowed. Only those students who completed the 
ethnic/racial question on both the SDQ and the answer sheet were included in the analyses. 

During the first fall administration, students were asked to indicate aN categories that 
they felt described them using essentially the same list of ethnic/racial categories found on the 
SDQ. (Note that although "Native Hawaiian or other Pacific Islander" was listed as a separate 
category, it was combined with "Asian or Asian American" for analysis purposes.) During the 
second administration, students were asked to describe themselves using two separate 
questions. First, they were to indicate their ethnicity as being "Hispanic or Latino" or "Not 
Hispanic or Latino." Then they were to indicate their race, using all of the remaining SDQ 
categories that applied. Table 1 presents both sets of questions. 

Results 

Shifts in Categories. For all analyses, the subcategories for "Hispanic" were collapsed 
into one category. A total of 349,608 students (93.1% of the SAT I test takers) responded to 
both the SDQ and the answer sheet questions in the first administration (see Table 2). Of 
these students, over 83% selected the same category both times. These numbers are shaded 
in Table 2. White (97.0%), African American (90.7%), and Asian American (93.2%) students 
tended to select a single category, while Hispanic and American Indian students spread out 
along a number of ethnic/racial categories. This is especially apparent in the American Indian 
category: only 35.4% of students choosing this category on the SDQ selected the category 
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again on the answer while 28.1% moved from "American Indian" to "White." 

Approximately 15,829 (4.5%) of the students shifted from a single category on the 
SDQ to more than one ethnic/racial category on the answer sheet. These numbers are 
presented in the bottom row of Table 2 (''Multiple"). Students who indicated they were 
"American Indian" or "Other" on the SDQ choose multiple categories more frequently than 
other groups: About 28.4% of students indicating "American Indian" choose multiple 
categories on the answer sheet and 28.3% of students indicating "Other" choose multiple 
categories on the answer sheet. While 4.5% may not seem like a large number, when taken in 
context of the 3 million students who take SAT Program tests each year, this percentage 
represents about 135,000 test takers. 

A further analysis examined the percent of students that moved from single to multiple 
ethnic/racial categories by region. This information is presented in Table 3. For this analysis, 
states were grouped into regions using the College Board-defined regions. In most regions, the 
shift from a single category response on the SDQ to a multiple category response on the 
answer sheet was less than 3%. For the Southwest, just over 4% of the student choose 
multiple ethnic/racial categories on the answer sheet. In the West, the percent of students 
moving from a single category to multiple categories was the highest of all regions, 7.5%. 

The volume for this fall administration is very high for the Western region (mostly due to 
California test takers) and thus, represents a high percentage of the total number of test 
takers. This undoubtedly influences the total shift seen in Table 2. It is possible that in other 
test administrations where the percentage of test takers from the West is lower, the shift from 
single- to multiple-categories will be lower than 4.5%. 

Test takers marked a wide variety of multiple categories and a total of 186 
combinations were found. However, combinations of two ethnic/racial categories represented 
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74% of students marking multiple categories. Table 4 displays the most frequently occurring 
combinations for this testing administration. Note that "White" and another racial category 
encompasses a substantial number of test takers. 

For the second administration, 243,820 students (92.9% of the SAT I test takers) 
responded to both the SDQ and the answer sheet questions (see Table 5). This set of 
questions appears to have caused confusion among test takers. For those students indicating 
they were Hispanic/Latino on both the answer sheet and the SDQ, over half of them selected 
"Other" as their race and about one-third selected no race category on the answer sheet. For 
those students indicating they were Hispanic/Latino on the answer sheet but not the SDQ, most 
selected the same race category on the answer sheet as they did on the SDQ. These cells are 
shaded in Table 5 (top table). Those students who indicated they were Hispanic on the SDQ 
but were not Hispanic on the answer sheet selected “Other,” “White,” or multiple race 
categories on the answer sheet. And the majority of the students who indicated they were not 
Hispanic on both the answer sheet and the SDQ selected the same race category on the answer 
sheet as they had on the SDQ. As seen in the first administration. White, African American, 
and Asian American students tended to select one category on the answer sheet, while 
American Indian students again selected a number of categories. These cells are shaded in 
Table 5 (bottom table). 

DIF analysis. Students from the first administration were used to compare differential 
item functioning (DIF) for groups formed using different ethnic/racial indicators. Two focal 
groups were formed: "Only" includes students who choose only a single ethnic/racial category 
on the answer sheet (e.g., only African American); "Plus" includes any student who choose a 
particular ethnic/racial category on the answer sheet and at least one other category (that is, 
they belonged to the "Multiple" category). 
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Mean Mantel-Haenszel DIF differences were computed using the "Original" (i.e., the 
particular ethnic/racial category in which students were placed based on their SDQ response) 
group as the reference group and the "Only," and the "Plus" groups as focal groups. For each 
ethnic/racial group the difference between each item (the "Original" minus "Only" or "Original" 
minus "Plus") was calculated and the mean using the absolute differences computed. 

Table 6 displays the differences for the items from the operational form given during the 
administration. As can be seen, the differences are very small, indicating that the DIF 
estimates are highly similar, irrespective of the particular group used. Additional analyses 
performed on pretest items (Table 7) produce similar results. Differences range from .01 to .14 
for the operational items and from .06 to .16 for the pretest items. 

These differences are quite reasonable, especially if compared to the differences seen 
when the same items are given to two different groups of "Original" students. For example, 
differences of .07 to .20 were seen in pretests that appeared in two different forms of the test 
at the same administration. These results are not surprising, however, in that the majority of 
students did not change ethnic/racial categories from the SDQ to the answer sheet. And of 
those students who indicated multiple categories, the majority still selected the ethnic/racial 
category they originally selected on the SDQ in addition to other category(ies). The 
ethnic/racial group showing the largest mean differences was "American Indian" and that group 
also displayed the greatest change in responses from SDQ to answer sheet. It should also be 
noted that the analyses with "American Indian" as the focal group also contained the smallest 
sample; the larger DIF differences may be an artifact of the small sample size. 

Summary 

These analyses provide preliminary information on the effect of allowing students to choose 
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multiple categories. While a small percentage of students across the country are likely to 
respond to multiple categories, the percentage of students is much larger in some regions (e.g., 
West, Southwest) than others. And taken in context of the number of students testing through 
the SAT Program, this small percentage could represent a substantial number of students. 

It appears that the two-level question used in the second administration is difficult for 
students to use. That is, the distinction between "race" and "ethnicity" may not be clear for 
some students. Thus, the single question used in the first administration seems to be the 
preferable one. 

While some recommendations can be made based on these data, it is apparent that 
additional work is needed to determine to full impact that changing demographics may present 
in current testing practices. There are also several issues that need further exploration. For 
example, the SDQ presents a more detailed list of categories for "Hispanic/Latino" (e.g., 

Mexican or Mexican American, Puerto Rican, etc.) but not for other categories, such as for 
"Asian or Asian American." Whether other ethnic/racial categories (e.g., "Asian or Asian 
American") also require a list of subcategories, and the impact of using subcategories versus a 
collapsed category in analyses such as DIF, should be studied further. 

In addition, the characteristics of the students who use the "Other" category are still 
largely unknown. Over the last 5 years, the percentage of students indicating "Other" on the 
SDQ has risen slightly (from 2% to 3%). It was expected that a good number of these 
students would move from "Other" to multiple categories. While the number of students 
selecting "Other" was reduced when they were able to choose more than one ethnic/racial 
category, there were still a substantial percentage of students who used "Other" to describe 
themselves. These students may feel that their ethnic/racial groups are not represented by the 
current list of categories. 
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Finally, results of this study indicated that there is little impact on DIF analyses when 
different definitions of ethnic/racial classifications are used compared to traditionally defined 
classifications. Until the percentage of students representing multiple ethnic/racial categories 
grows, it is unlikely that current testing practices need to be modified. However, the changing 
demographics of these test takers should continued to be monitored and tracked to determine 
possible impact in the future. 



Table 1. Ethnic/ Racial Questions 



student Descriptive Questionnaire question: 



1. How do you describe yourself? (Mark only one.) 

American Indian or Alaskan Native 
Asian, Asian American, or Pacific Islander 
African American or Black 
Hispanic or Latino background: 

Mexican or Mexican American 
Puerto Rican 

Latin American, South American, Central 
American, or other Hispanic or Latino 
White 
Other 



First Administration question: 



1. How do you describe yourself? Mark all choices that apply: 

American Indian or Alaskan Native 
Asian or Asian American 
African American or Black 
Hispanic or Latino background: 

Mexican or Mexican American 
Puerto Rican 

Latin American, South American, Central 
American, or other Hispanic or Latino 
Native Hawaiian or other Pacific Islander 
White 
Other 



Second Administration questions: 



1. How do you describe yourself? Choose only one: 

Hispanic or Latino 
Not Hispanic or Latino 



2. How do you describe yourself? Mark aii choices that apply: 

American Indian or Alaskan Native 

Asian or Asian American 

African American or Black 

Native Hawaiian or other Pacific Islander 

White 

Other 
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Table 2. Student Responses to Ethnic/ Racial 
Categories on SDQ and SAT I Answer Sheet 

(First Administration) 



SDQ Response 



Answer 

Sheet 

BesDonse 


American 

Indian 


Asian 

American 


African 

American 


Hispanic/ 

Latino 


White 


Other 


American 

Indian 


736 

(35.4) ^ 








Asian 

American 


70 

(3.4) 


31,696 

(93.2) 








506 

(5.7) 


African 

American 


54 

(2.6) 




23,956 

(90-7) 






238 

(2.7) 


Hispanic/ 

Latino 








21,266 

(87.2) 






White 


585 

(28.1) 






291 

(1.2) 


211,778 

(97.0) 


o 

CO 


Other 


32 

(1.5) 


480 

(1.4) 


265 

(1.0) 






4,086 

(46,3) 
















Multiple 


591 

(28-4) 


1,572 

{4,6} 


1,785 

{6:8} 


2,584 

(10.6) 


4,185 

(1.9) 


2,496 

(28,3) 



Note. Numbers in parentheses are percents. Empty cells mean less than 1% of students indicated 
membership in that ethnic/racial group. 

Above table is based on 349,608 test takers. These students represented about 93.1% of students 
taking a fall 1998 SAT I who completed the Student Descriptive Questionnaire (SDQ) and the answer sheet 
(A/S) question about ethnicity/race. 
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Table 3. Percent of Students Choosing Multiple 
Categories, by Region 



Region 


Percent of 
Students 


New England 
Middle States 
South 

Midwest ^ 

Southwest 

West ; v; : : 


2.67 
2.84 , 
2.90 
2.51 
4.06 
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Table 4. Most Frequently Occurring Ethnic/Racial 

Combinations 



Frequency 


Categories 


2,073 


White - American Indian 


1,921 


White - Asian American 


1,706 


White - Other 


1,346 


White - Other Latino 


1,160 


White - Mexican 


813 


White - African American 


526 


White - Puerto Rican 


484 


American Indian - African American 


412 


Asian American - Other 


406 


Mexican - Other Latino 


284 


African American - Other 


269 


African American - Other Latino 


263 


Asian American - African American 
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Table 5. Student Responses to Ethnic/Racial Categories 
on SDQ and SAT I Answer Sheet 
(Second Administration) 



Hispanic = Yes 



A/S 

Response 


American 

Indian 


Asian 

American 


African 

American 


Hispanic/ 

Latino 


White 


Other 


American 

Indian 


37 

(42.5) 












Asian 

American 




174 

(72.2) 










African 

American 




3 

(1.2) 


284 

(43-4) 






73 

(11-2) 


White 


2 

(2.3) 


4 

(1.7) 


15 

(2.3) 


1,159 

(6.9) 


532 

(60.2) 


77 

(11-8) 


Other 


10 

(11.5) 


11 

(4.6) 


160 

(24.4) 


8,968 

(53.5) 


79 

(8.9) 


176 

(27.0) 
















Multiple 


33 

(37.9) 


45 

(18.7) 


105 

(16.0) 


1,239 
(74) 


200 

(22.6) 


249 

(38.2) 


None 


5 

(5.7) 


4 

(1-7) 


84 

(12.8) 


2,137 

(30.8) 


45 

(5.1) 


42 

(6.4) _ 



Note. Numbers in parentheses are percents. Empty cells mean less than 1 % of students indicated 
membership in that ethnic/racial group. 



Hispanic = No 

SDQ Response 



A/S 

Response 


American 

Indian 


Asian 

American 


African 

American 


Hispanic/ 

Latino 


White 


Other 


American 

Indian 


414 

(33.5) 












Asian 

American 


35 

(2.8) 


14,423 

(90.2) 




19 

(3.8) 




257 

(5.0) 


African 

American 


48 

(3.9) 




17,492 

(90.6) 


53 

(10.7) 




244 

(4.8) 


White 


344 

(27.8) 






171 

(34.4) 


120,318 
(96.4) . 


819 

(16.0) 


Other 


21 

(1.7) 


339 

(2.1) 


236 

(1.2) 


188 

(37.8) 




2,353 

(46.1) 
















Multiple 


374 

(30.2) 


1,005 

(6.3) 


1,407 

(7.3) 


54 

(10-9j 


2,936 

(2.4) 


1,381 

(27.1) 


1 None 








9 

(1.8) 







Note. Numbers in parentheses are percents. Empty cells mean less than 1% of students indicated 
membership in that ethnic/racial group. 
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Table 6. Mean Mantel-Haenszel D-DIF Differences 

for Operational Items 



Comparison 


Group 


Verbal 


Math 






(# items = 78) 


(# items = 60) 




Only 


.01 


.02 


African AmericanA/Vhite 


Plus 


.02 


.02 




Only 


.03 


.03 


HispanicA/Vhite 


Plus 


.03 


.03 




Only 


.02 


.03 


Asian AmericanA/Vhite 








- 


Plus 


.03 


.05 




Only 


.14 - 


.14 


American Indian/White 


Plus 


.14 


.12 
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Table 7. Mean Mantel-Haenszel D-DIF Differences 

for Pretest Items 



Pretest 


African 

American/ 

White 


Hispanic/ 

White 


Asian 

American/ 

White 




Only 


Plus 


Only 


Plus 


Only 


Plus 


Verbal 

1 (# of Items = 35) 


.06 


.07 


.11 


.08 


.08 


.09 


2 (# of items = 35) 


.07 


.07 


.09 


.12 


.10 


.10 


3 (# of items = 35) 


.11 


.10 


.13 


.12 


.10 


.10 


4 (# of Items = 35) 


.07 


.07 


.14 


.11 


.08 


.09 


5 (# of Items = 35) 


.08 


.08 


.11 


.10 


.07 


.09 


6 (# of items = 35) 


.07 


.08 


.13 


.12 


.07 


.10 


7 (# of Items = 30) 


.09 


.07 


.16 


.12 


.14 


.15 


8 (# of Items = 30) 


.10 


.09 


.14 


.13 


.12 


.12 


9 (# of Items = 35) 


.06 


.07 


.10 


.08 


.08 


.07 


10 (# of Items = 30) 


.07 


.07 


.12 


.12 


.08 


.09 


11 (# of Items = 30) 


.08 


.07 


.10 


.08 


.06 


.05 


12 (# of Items = 30) 


.09 


.07 


.09 


.08 


.08 


.08 


Math 

1 (# of Items = 25) 


.09 


.07 


.12 


.12 


.13 


.12 


2 (# of Items = 25) 


.11 


.06 


.16 


.11 


.10 


.11 


3 (# of items = 25) 


.06 


.06 


.08 


.07 


.12 


.11 


4 (# of items = 25) 


.16 


.09 


.14 


.14 


.09 


.12 


5 (# of Items = 25) 


.06 


.05 


.14 


.13 


.12 


.10 


6 (# of Items = 25) 


.10 


.10 


.13 


.15 


.10 


.11 



Note. Sample sizes too small to perform “American Indian” analyses. 
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